Non-transitory computer-readable recording medium, estimation method, and information processing device

ABSTRACT

A non-transitory computer-readable recording medium storing an estimation program that causes a processor included in a computer to execute a process. The process includes receiving a message of an estimation target of a forwarding destination, searching past messages for a message similar to the received message, estimating, when there is no similar message among the past massage, the forwarding destination of the message using a model of estimating the forwarding destination based on a keyword that is identified based on a word included in the received message, and outputting a forwarding destination of the searched similar message or the estimated forwarding destination.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority of theprior Japanese Patent Application No. 2020-197099, filed on Nov. 27,2020, the entire contents of which are incorporated herein by reference.

FIELD

The embodiment discussed herein is related to a non-transitorycomputer-readable recording medium, an estimation method, and aninformation processing device.

BACKGROUND

In a case where an error message is received from a system, a person incharge of operation of a system to be monitored forwards the errormessage to a department that handles the trouble. In recent years, sincethe number of systems has increased due to advances in cloud technologyand the load on the person in charge of operation has increased due tothe increase in the number of error messages and forwardingdestinations, reducing the load on the person in charge of operation hasbeen in demand.

There has been known a technique of performing learning/clustering basedon anomalous cases and classifying and presenting new observation dataas a conventional technique of supporting the work of forwarding errormessages for reducing such load on the person in charge of operation.

Japanese Laid-open Patent Publication No. 2009-31810 and JapaneseLaid-open Patent Publication No. 2013-41448 are disclosed as relatedart.

SUMMARY

According to an aspect of the embodiments, a non-transitorycomputer-readable recording medium storing an estimation program thatcauses a processor included in a computer to execute a process, theprocess includes: receiving a message of an estimation target of aforwarding destination; searching past messages for a message similar tothe received message; estimating, when there is no similar message amongthe past massage, the forwarding destination of the message using amodel of estimating the forwarding destination based on a keyword thatis identified based on a word included in the received message; andoutputting a forwarding destination of the searched similar message orthe estimated forwarding destination.

The object and advantages of the invention will be realized and attainedby means of the elements and combinations particularly pointed out inthe claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and arenot restrictive of the invention.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an exemplary functionalconfiguration of an information processing device according to anembodiment;

FIG. 2A is an explanatory diagram exemplifying data contents of a pastmessage storage unit;

FIG. 2B is an explanatory diagram exemplifying data contents of akeyword dictionary storage unit and a non-keyword dictionary storageunit;

FIG. 2C is an explanatory diagram exemplifying data contents of akeyword vector storage unit;

FIG. 3 is a flowchart illustrating an exemplary operation of a learningphase;

FIG. 4 is a flowchart illustrating an exemplary word combination processin the learning phase;

FIG. 5 is an explanatory diagram illustrating an outline of the wordcombination process in the learning phase;

FIG. 6 is an explanatory diagram illustrating exemplary data of learningmodel generation;

FIG. 7 is a flowchart illustrating an exemplary operation of theinformation processing device according to the embodiment;

FIG. 8 is a flowchart illustrating an exemplary operation of anestimation phase;

FIG. 9 is a flowchart illustrating an exemplary word combination processin the estimation phase;

FIG. 10 is an explanatory diagram illustrating an outline of the wordcombination process in the estimation phase;

FIG. 11 is an explanatory diagram illustrating exemplary output of aforwarding destination;

FIG. 12 is an explanatory diagram illustrating exemplary output of theforwarding destination;

FIG. 13 is an explanatory diagram illustrating exemplary keyword data;

FIG. 14 is an explanatory diagram illustrating an exemplary outputresult; and

FIG. 15 is an explanatory diagram illustrating an exemplary computerconfiguration.

DESCRIPTION OF EMBODIMENTS

The conventional technique described above has a problem that it isdifficult to classify an unknown error message, which is not similar tothe past cases, into an appropriate forwarding destination.

In one aspect, it aims to provide an estimation program, an estimationmethod, and an information processing device capable of classifying amessage into an appropriate forwarding destination.

Hereinafter, an estimation program, an estimation method, and aninformation processing device according to an embodiment will bedescribed with reference to the drawings. Configurations having the samefunctions in the embodiment are denoted by the same reference signs, andredundant description will be omitted. Note that the estimation program,the estimation method, and the information processing device describedin the following embodiment are merely examples, and do not limit theembodiment. Furthermore, each of the embodiments below may beappropriately combined unless otherwise contradicted.

FIG. 1 is a block diagram illustrating an exemplary functionalconfiguration of the information processing device according to theembodiment. As illustrated in FIG. 1, an information processing device 1is a device that displays and outputs, on a display or the like, aforwarding destination of an error message (hereinafter, also referredto as a new message) received from a monitored system 2 to present it toa person in charge of operation (user). In this manner, the informationprocessing device 1 supports the work of the user who forwards the errormessage to a department that handles a trouble of the monitored system2. For example, a personal computer (PC) or the like may be applied asthe information processing device 1.

Specifically, for example, the information processing device 1 includesa message reception unit 10, an estimation unit 11, a storage unit 12,an output unit 13, and a learning unit 14.

The message reception unit 10 is a processing unit that receives newmessages collected and distributed by a message transmission unit 20 ofthe monitored system 2 from a Syslog, application log, MWlog,performance log, and the like. Specifically, for example, the messagereception unit 10 includes a new message reception unit 101 and afiltering execution unit 102.

The new message reception unit 101 receives new messages distributed bythe monitored system 2 via a network such as a local area network (LAN).The new message reception unit 101 outputs the received new messages tothe filtering execution unit 102.

The filtering execution unit 102 is a processing unit that executesfiltering of the new messages received by the new message reception unit101 on the basis of a preset rule. For example, the filtering executionunit 102 excludes a new message from the forwarding target in a casewhere a flag indicating a level of importance included in the newmessage, a character string included in the message contents, or thelike does not satisfy predetermined conditions. As an example, thefiltering execution unit 102 excludes a new message with a level ofimportance less than a threshold value or not including a characterstring such as “urgent” from the forwarding target as a message relatedto an error that does not need to be dealt with.

The estimation unit 11 is a processing unit that estimates a forwardingdestination of the new message to be forwarded received by the messagereception unit 10. Specifically, for example, the estimation unit 11includes a new message preprocessing unit 111, a similar messagedetermination unit 112, a keyword determination unit 113, a similar wordsearch unit 114, a similar word storage unit 115, a forwardingdestination estimation unit 116, and a forwarding destination outputunit 117.

The new message preprocessing unit 111 is a processing unit thatpreprocesses a new message to be forwarded. Specifically, for example,the new message preprocessing unit 111 applies, to a new message, aregular expression or the like representing a target to be matched,thereby extracting a predetermined character string. For example, thenew message preprocessing unit 111 divides a new message into respectivewords included in a message title, body, and the like. Then, the newmessage preprocessing unit 111 vectorizes the new message on the basisof the divided words using a publicly known vectorization method such asthe term frequency-inverse document frequency (tf-idf).

The similar message determination unit 112 is a processing unit thatcompares the new message preprocessed by the new message preprocessingunit 111 with respective past messages stored in the past messagestorage unit 121 that stores past message data to make a determinationof a similar message. Specifically, for example, the similar messagedetermination unit 112 compares the new message vectorized using tf-idfor the like with the past message to obtain cosine similarity. Then,when the obtained cosine similarity is equal to or higher than apredetermined threshold value, the similar message determination unit112 determines that the new message and the past message are similar.

The keyword determination unit 113 is a processing unit that determineswhether or not each word included in a new message is a specifickeyword. Specifically, for example, the keyword determination unit 113refers to a keyword dictionary storage unit 122 that stores a keyworddictionary in which a plurality of keywords is present, and when eachword included in the new message is included in the keyword dictionary,identifies the word as a keyword.

Furthermore, when each word included in the new message is not includedin the keyword dictionary, the keyword determination unit 113 refers toa non-keyword dictionary storage unit 123 that stores a non-keyworddictionary in which a plurality of non-keywords not identified as akeyword is present, and determines whether or not each word is includedin the non-keyword dictionary. Here, it is assumed that the keyworddetermination unit 113 does not use a word included in the non-keyworddictionary as a keyword.

For a word not included in the non-keyword dictionary, the keyworddetermination unit 113 performs searching to find out whether or not asimilar word similar to the word is included in the keyword dictionaryusing the similar word search unit 114. Here, in a case where a similarword similar to the word is included in the keyword dictionary, thekeyword determination unit 113 identifies the similar word included inthe keyword dictionary, which has been found through the searching, as akeyword.

The similar word search unit 114 is a processing unit that searches thekeyword dictionary stored in the similar message determination unit 112for a similar word similar to the word instructed to be searched for bythe keyword determination unit 113. Specifically, for example, thesimilar word search unit 114 vectorizes meanings of words, and by thevectorization, it searches for a similar word similar to the wordaccording to calculation of closeness of meanings between words or usinga publicly known method, such as Word2vec, which makes it possible toevaluate a combination of multiple words by adding or subtracting thewords themselves. The similar word search unit 114 returns the similarword obtained through the search to the keyword determination unit 113,and stores it in the similar word storage unit 115. The similar wordstorage unit 115 is a memory that stores similar words obtained throughthe search performed by the similar word search unit 114.

The forwarding destination estimation unit 116 is a processing unit thatestimates a forwarding destination using a forwarding destinationestimation model that estimates a forwarding destination stored in theforwarding destination estimation model storage unit 125 on the basis ofthe keyword identified by the similar word storage unit 115 based on theword included in the new message. The forwarding destination estimationmodel (also called a learning model) is a model generated by thelearning unit 14 based on machine learning using past messages asteaching data in such manner that an estimation result of a forwardingdestination is output by inputting information associated with a keywordincluded in a message. The forwarding destination estimation unit 116inputs, to the forwarding destination estimation model, informationassociated with the keyword identified by the similar word storage unit115, thereby obtaining an estimation result of the forwardingdestination.

The forwarding destination output unit 117 is a processing unit thatoutputs the estimation result of the forwarding destination estimated bythe estimation unit 11. Specifically, for example, when there is amessage similar to the new message in the past messages stored in thepast message storage unit 121, the forwarding destination output unit117 reads the forwarding destination of the similar past message fromthe past message storage unit 121, and outputs it. Furthermore, whenthere is no message similar to the new message in the past messages, theforwarding destination output unit 117 outputs a forwarding destinationestimated by the forwarding destination estimation unit 116 from the newmessage.

The storage unit 12 is a storage device, such as a memory, which storesvarious kinds of information. Specifically, for example, the storageunit 12 includes the past message storage unit 121, the keyworddictionary storage unit 122, the non-keyword dictionary storage unit123, a keyword vector storage unit 124, and the forwarding destinationestimation model storage unit 125.

The past message storage unit 121 stores respective cases (errormessages) that have occurred in the past in the monitored system 2 aspast messages with forwarding destinations.

FIG. 2A is an explanatory diagram exemplifying data contents of the pastmessage storage unit 121. As illustrated in FIG. 2A, the past messagestorage unit 121 stores data of message, recommend, label, key1, key2,key3, and so on, for each of past messages. Here, the item “message” ismessage contents (body). The item “recommend” is contents of aforwarding destination. The item “label” is a label of contents of aforwarding destination. The items “key1”, “key2”, “key3”, and so on arelists of keywords included in the message contents.

The keyword dictionary storage unit 122 stores a keyword dictionaryindicating a plurality of keywords registered in advance. Thenon-keyword dictionary storage unit 123 stores a non-keyword dictionaryindicating a plurality of non-keywords registered in advance.

FIG. 2B is an explanatory diagram exemplifying data contents of thekeyword dictionary storage unit 122 and the non-keyword dictionarystorage unit 123. As illustrated in FIG. 2B, the keyword dictionarystorage unit 122 stores each keyword. The non-keyword dictionary storageunit 123 stores each non-keyword (notkeyword).

At the time of machine learning using past messages as teaching data,the non-keyword dictionary storage unit 123 stores keyword vectorsobtained by vectorizing respective past messages.

FIG. 2C is an explanatory diagram exemplifying data contents of thekeyword vector storage unit 124. As illustrated in FIG. 2C, the keywordvector storage unit 124 stores data obtained by dividing past messages(“abc”, “def”, etc.) into respective words (“a”, “b”, etc.) andvectorizing them.

The forwarding destination estimation model storage unit 125 stores datasuch as parameters related to the forwarding destination estimationmodel generated by the learning unit 14.

The output unit 13 is a processing unit that outputs, to the user,information associated with the forwarding destination of the newmessage estimated by the estimation unit 11. Specifically, for example,the output unit 13 includes a message display unit 131. The messagedisplay unit 131 displays, on a display, the estimation result of theforwarding destination output by the forwarding destination output unit117.

The learning unit 14 is a processing unit that generates (learns) aforwarding destination estimation model on the basis of machine learningusing past messages as teaching data. Specifically, for example, thelearning unit 14 includes a past message input unit 141, a past messagepreprocessing unit 142, a keyword classification unit 143, a forwardingdestination learning unit 144, and a forwarding destination estimationmodel output unit 145.

The past message input unit 141 is a processing unit that receives cases(error messages) that have occurred in the past in the monitored system2 with forwarding destinations by input from the user. The past messageinput unit 141 stores data of the received past messages in the pastmessage storage unit 121.

The past message preprocessing unit 142 is a processing unit that readsthe past messages stored in the past message storage unit 121 asteaching data for supervised machine learning, and performspreprocessing. Specifically, for example, in a similar manner to the newmessage preprocessing unit 111, a past message is divided intorespective words included in a message title, body, and the like.

The keyword classification unit 143 is a processing unit that classifieswords included in a past message into keywords and non-keywords withrespect to the past messages preprocessed by the past messagepreprocessing unit 142.

For example, the keyword classification unit 143 presents, on a displayor the like, each word divided from the past message to the user, andreceives a keyword or non-keyword selection instruction for each word.In response to this selection instruction, the keyword classificationunit 143 classifies the words included in the past message into keywordsand non-keywords. Furthermore, the keyword classification unit 143 mayrefer to the word itself as a keyword (or non-keyword) or keywordsetting data in which conditions for the word as a keyword (ornon-keyword) are set in advance to classify the words included in thepast message into keywords and non-keywords.

The keyword classification unit 143 stores the classified keywords andnon-keywords in the keyword dictionary storage unit 122 and thenon-keyword dictionary storage unit 123. As a result, the keywordclassification unit 143 creates a keyword dictionary and a non-keyworddictionary from past messages.

The forwarding destination learning unit 144 is a processing unit thatperforms machine learning of a forwarding destination estimation modelin such a manner that forwarding destinations of the keywords arecalculated from the keywords classified from the past messages, with theforwarding destinations assigned to the past messages as correctanswers.

Specifically, for example, the forwarding destination learning unit 144vectorizes the keywords classified from the past messages, and storesthem in the keyword vector storage unit 124. Then, the forwardingdestination learning unit 144 learns parameters of each layer of theforwarding destination estimation model in such a manner that the labelindicating the forwarding destination assigned to the past message isoutput from the output layer when a keyword vector is input to the inputlayer of the forwarding destination estimation model.

The forwarding destination estimation model output unit 145 is aprocessing unit that stores the forwarding destination estimation modelmachine-learned by the forwarding destination learning unit 144 in theforwarding destination estimation model storage unit 125. Specifically,for example, the forwarding destination estimation model output unit 145stores, in the forwarding destination estimation model storage unit 125,the parameters of each layer in the forwarding destination estimationmodel obtained by machine learning.

Here, operation of the learning phase related to machine learning by thelearning unit 14 will be described. FIG. 3 is a flowchart illustratingan exemplary operation of the learning phase.

As illustrated in FIG. 3, when the process starts, the past messagepreprocessing unit 142 reads data of a past message stored in the pastmessage storage unit 121 (S10). Next, the past message preprocessingunit 142 divides the sentence of the read data of the past message intowords (S11).

Next, the keyword classification unit 143 determines, for each dividedword, whether or not a keyword in the keyword setting data is included(S12). When a keyword is included (Yes in S12), the keywordclassification unit 143 stores the word in the keyword dictionary (S13).When the keyword is not included (No in S12), the keyword classificationunit 143 stores the word in the non-keyword dictionary (S14).

Note that, in the case of storing words in the keyword dictionary, thekeyword classification unit 143 may store the words after integratingthem through a word combination process. FIG. 4 is a flowchartillustrating an exemplary word combination process in the learningphase. FIG. 5 is an explanatory diagram illustrating an outline of theword combination process in the learning phase.

As illustrated in FIGS. 4 and 5, when the process starts, the pastmessage preprocessing unit 142 reads data of a past message stored inthe past message storage unit 121 (S20). Next, the past messagepreprocessing unit 142 divides the sentence of the read data of the pastmessage into words (S21).

Next, the keyword classification unit 143 extracts English words fromthe divided words (S22). Next, the keyword classification unit 143determines whether or not the extracted English words are included inthe keywords in the keyword setting data (S23). When the extractedEnglish words are included in the keyword setting data (Yes in S23), thekeyword classification unit 143 replaces the English words with thekeywords of the keyword setting data (S24). When the extracted Englishwords are not included in the keyword setting data (No in S23), thekeyword classification unit 143 skips the processing of S24, andproceeds to the processing of S25.

Next, the keyword classification unit 143 determines whether or not allthe English words have been processed (S25), and returns the process toS23 if there is an unprocessed English word (No in S25).

If all the English words have been processed (Yes in S25), the keywordclassification unit 143 deletes duplicate keywords generated as a resultof the replacement (S26), and terminates the process. As a result, inthe example of FIG. 5, a keyword [Error, Connection_Refused] is storedin the keyword dictionary from a past message D1 (S13).

Since there is no regularity in an input document in general documentclassification based on machine learning, it is difficult to graspcharacteristics of the document unless the document is separated intoword units by morphological analysis. However, in error messages, theremay be a case where recognition of long character strings is effectiveby using types (relevance between the character string and the messagebecomes higher as the length of the recognizable character stringbecomes longer, which makes it easier to be recognized).

At the time of vectorizing keywords, the keywords have been dividedbased on a specified rule such as a regular expression, and thus thewords included in the keyword dictionary created from the past messageD1 have also been divided on the basis of the rule. For example, when akeyword in the message is English, the keyword is to be divided in wordunits separated by half-width spaces.

Here, since an error message has a tendency of having a certain type,the longer a keyword is, the easier it is to recognize the message fromthe keyword. Therefore, when the keyword for the English message is ashort keyword on a word-by-word basis, there may arise a problem thatthe relevance to the original message is lowered and the accuracy islowered.

For example, in a case where a keyword is “Connection Refused”, thekeyword is divided into “Connection” and “Refused” after beingvectorized, thereby lowering the relevance between the keyword and themessage.

However, in the information processing device 1, the keyword to bestored in the keyword dictionary remains to be “Connection_Refused” bythe word combination process described above performed.

Returning to FIG. 3, the learning unit 14 determines whether or not allthe past messages D1 read from the past message storage unit 121 havebeen processed (S15). When there is an unprocessed past message D1 (Noin S15), the learning unit 14 returns the process to S11.

When all the past messages D1 have been processed (Yes in S15), theforwarding destination learning unit 144 vectorizes the keywords of thepast messages D1 (S16), and stores the keyword vectors of the pastmessages D1 in the keyword vector storage unit 124 (S17). Next, theforwarding destination learning unit 144 creates a learning model of theforwarding destination for the keyword (S18), and terminates theprocess.

FIG. 6 is an explanatory diagram illustrating exemplary data of learningmodel generation. As illustrated in FIG. 6, in generation of a learningmodel, a keyword vector D10 obtained by vectorizing the keywordidentified by word division and a label D11 obtained by vectorizing theforwarding destination assigned to the past message D1 are prepared fromthe past message D1. The forwarding destination learning unit 144generates (learns) a learning model in such a manner that, in a casewhere the keyword vector D10 is input to the input layer of the learningmodel, the label D11 is output from the output layer.

FIG. 7 is a flowchart illustrating an exemplary operation of theinformation processing device 1 according to the embodiment. Asillustrated in FIG. 7, the message transmission unit 20 transmits a newmessage (error message) newly generated from the monitored system 2(S30).

Next, the new message reception unit 101 receives the new message fromthe monitored system 2 (S31). Next, the filtering execution unit 102filters the received new message by event monitoring performed at apredetermined timing (e.g., at intervals of several seconds) (S32).

Next, the new message preprocessing unit 111 performs preprocessing(e.g., word division etc.) on the new message for estimating aforwarding destination (S33). Next, the similar message determinationunit 112 determines whether or not a message similar to the new messageis found in the past messages of the past message storage unit 121(S34).

When a similar message is found (Yes in S34), the forwarding destinationoutput unit 117 reads the forwarding destination of the similar messagefrom the past message storage unit 121, and outputs it (S35).

When no similar message is found (No in S34), the keyword determinationunit 113 identifies a keyword from the new message. Next, the forwardingdestination estimation unit 116 estimates a forwarding destination usinga machine learning model (forwarding destination estimation model) basedon the identified keyword (S36). Next, the forwarding destination outputunit 117 outputs the forwarding destination of the estimation result ofthe forwarding destination estimation unit 116 (537).

Next, the message display unit 131 displays the forwarding destinationoutput in S35 or S37 on the monitoring screen of the display (S38), andterminates the process.

Here, an estimation phase related to the estimation of the forwardingdestination of the new message will be described. FIG. 8 is a flowchartillustrating an exemplary operation of the estimation phase.

As illustrated in FIG. 8, when the process starts, the similar messagedetermination unit 112 refers to past message data of the past messagestorage unit 121 to calculate similarity (e.g., cosine similarity) tothe new message (S40).

Next, the similar message determination unit 112 determines whether ornot the calculated similarity exceeds a threshold value (S41). When thesimilarity exceeds the threshold value (Yes in S41), the message displayunit 131 displays the forwarding destination of the similar message topresent it to the user (S42).

When the calculated similarity does not exceed the threshold value (Noin S41), the new message preprocessing unit 111 divides the new messageinto words (S43). Next, the keyword determination unit 113 determineswhether or not a divided word is included in the keyword dictionary ofthe keyword dictionary storage unit 122 (S44).

When it is included in the keyword dictionary (Yes in S44), the keyworddetermination unit 113 determines that the divided word is a keyword(S45). When it is not included in the keyword dictionary (No in S44),the keyword determination unit 113 determines whether or not the dividedword is included in the non-keyword dictionary of the non-keyworddictionary storage unit 123 (S46).

when it is included in the non-keyword dictionary (Yes in S46), thekeyword determination unit 113 proceeds to the processing of S50 withoutusing the divided word as a keyword.

When it is not included in the non-keyword dictionary (No in S46), thesimilar word search unit 114 calculates a similar word of the dividedword (S47), and determines whether or not the calculated similar word isincluded in the keyword dictionary (S48).

When the similar word is included in the keyword dictionary (Yes inS48), the keyword determination unit 113 determines that the similarword is a keyword (S49). When the similar word is not included in thekeyword dictionary (No in S48), the keyword determination unit 113proceeds to the processing of S50 without using the similar word as akeyword.

Next, the keyword determination unit 113 determines whether or not allthe divided words have been processed (S50), and returns the process toS44 when there is an unprocessed word (No in S50).

When all the words have been processed (Yes in S50), the forwardingdestination estimation unit 116 vectorizes the word determined to be akeyword (S51). Next, the forwarding destination estimation unit 116reads the forwarding destination estimation model from the forwardingdestination estimation model storage unit 125, and inputs the vectorizedkeyword vector to the model, thereby calculating a forwardingdestination (S52).

Next, the message display unit 131 receives the calculation result ofthe forwarding destination estimation unit 116 from the forwardingdestination output unit 117, and displays, on the display, the keywordused in the calculation and the calculated forwarding destination (S53).

Note that the estimation unit 11 may perform the word combinationprocess in a similar manner to the machine learning of the learning unit14. FIG. 9 is a flowchart illustrating an example of the wordcombination process in the estimation phase. FIG. 10 is an explanatorydiagram illustrating an outline of the word combination process in theestimation phase.

As illustrated in FIGS. 9 and 10, when the process starts, the newmessage preprocessing unit 111 reads a new message D2 (S60). Next, thenew message preprocessing unit 111 divides the read new message D2 intowords (S61).

Next, the keyword determination unit 113 extracts English words from thedivided words (S62), and determines whether or not the extracted Englishwords are included in the keywords of the keyword dictionary (S63).

When an extracted English word is included in the keywords of thekeyword dictionary (Yes in S63), the keyword determination unit 113stores the pair of the English word and keyword in a keyword candidatelist (S64). When the extracted English word is not included in thekeywords of the keyword dictionary (No in S63), the keyworddetermination unit 113 proceeds to the processing of S65 without storingthe English word in the candidate list.

Next, the keyword determination unit 113 determines whether or not allthe English words have been processed (S65), and returns the process toS63 when there is an unprocessed English word (No in S65).

When all the English words have been processed (Yes in S65), the keyworddetermination unit 113 integrates (concatenates) English words havingthe same keyword included in the candidate list (S66), and terminatesthe process. As a result, in the example of FIG. 10, a keyword [Error,Connection_Refused] is identified from the new message D2.

For example, when a word included in the new message D2 is “ConnectionRefused”, the keyword is divided into “Connection” and “Refused” afterbeing vectorized as it is, thereby lowering the relevance between thekeyword and the message.

However, in the information processing device 1, the keyword to beidentified from the new message D2 remains to be “Connection_Refused” bythe word combination process described above performed. Therefore, inthe information processing device 1, the relevance between the characterstring and the message becomes higher as the length of the recognizablecharacter string (keyword) becomes longer, and the effect of increasingestimation accuracy of the forwarding destination may be expected.

Here, an example of the output (display) of the forwarding destinationin S42 and S53 will be specifically described. FIGS. 11 and 12 areexplanatory diagrams illustrating exemplary output of the forwardingdestination. Specifically, for example, FIG. 11 is a diagramillustrating an example of the output (display) of the forwardingdestination in S42. FIG. 12 is a diagram illustrating an example of theoutput (display) of the forwarding destination in S53.

As illustrated in FIG. 11, in S42, forwarding destination results R1 toR3 indicating forwarding destinations of similar messages similar to thenew message are output in order of similarity. Here, in the forwardingdestination results R1 to R3, an item “message” indicates contents of asimilar message. An item “similarity” indicates similarity to the newmessage. An item “action” indicates a forwarding destination. Bychecking the contents of the forwarding destination results R1 to R3,the user is enabled to classify the forwarding destination of the newmessage.

As illustrated in FIG. 12, when there is no similar message similar tothe new message D2, the keyword is identified, the forwardingdestination is estimated using the forwarding destination estimationmodel, and the estimation result is displayed. Here, the keyworddetermination unit 113 obtains a similar word for a new word in the casewhere there is no keyword matching the word of the new message D2 in thekeyword dictionary (S47). Then, when a similar word is included in thekeyword dictionary, the keyword determination unit 113 uses the similarword as a keyword.

FIG. 13 is an explanatory diagram illustrating exemplary keyword data.Keyword data K1 of FIG. 13 indicates words in the new message D2.Keyword data K2 indicates words identified as keywords from the newmessage D2.

The forwarding destination estimation unit 116 vectorizes the keyworddata K2, and inputs it to the forwarding destination estimation model,thereby obtaining a forwarding destination (552). The forwardingdestination output unit 117 outputs, for example, the top threecandidates of the obtained forwarding destinations together with thekeywords used in the estimation of the forwarding destination. As aresult, the message display unit 131 displays, on the display, the topthree candidates of the forwarding destinations together with thekeywords (S53).

FIG. 14 is an explanatory diagram illustrating an example of the outputresult. Note that FIG. 14 exemplifies the output results of the top twocandidates for the forwarding destination. As illustrated in FIG. 14, aforwarding destination result R10 includes a forwarding destinationindicated by “action” and a score (probability) calculated by theforwarding destination estimation model. As a result, the user isenabled to check the forwarding destination of the new message D2 andits score.

The output result includes the forwarding destination result R10 usedfor the estimation of the forwarding destination. Specifically, forexample, the forwarding destination result R10 includes a score(explanations) for each keyword. As a result, the user is enabled toeasily check the keyword used at the time of estimating the forwardingdestination of the new message D2.

As described above, the information processing device 1 receives the newmessage D2 of the estimation target of the forwarding destination fromthe monitored system 2. The information processing device 1 searches thepast messages of the past message storage unit 121 for a message similarto the received new message D2. When there is no similar message in thepast message storage unit 121, the information processing device 1estimates a forwarding destination of the new message D2 using a modelfor estimating the forwarding destination stored in the forwardingdestination estimation model storage unit 125 based on the keywordidentified based on the word included in the received new message D2.The information processing device 1 outputs the forwarding destinationor the estimated forwarding destination of the searched similar message.

As a result, in a case where the new message D2 is an unknown messagewithout a similar message in the past message storage unit 121, theinformation processing device 1 is capable of classifying the messageinto the forwarding destination estimated using the keyword identifiedfrom the words included in the message. Therefore, according to theinformation processing device 1, it is also possible to classify anunknown message into an appropriate forwarding destination.

Furthermore, when a word included in the new message D2 is included inthe keyword dictionary stored in the keyword dictionary storage unit122, the information processing device 1 identifies the word as akeyword. As a result, according to the information processing device 1,it is possible to identify, among words included in the new message D2,a word included in the keyword dictionary stored in the keyworddictionary storage unit 122 as a keyword to be used in the model ofestimating a forwarding destination. Therefore, according to theinformation processing device 1, it is possible to estimate, even for anunknown message without a similar message in the past message storageunit 121, a forwarding destination using a known keyword included in themessage.

Furthermore, when a word included in the new message D2 is not includedin the keyword dictionary stored in the keyword dictionary storage unit122, the information processing device 1 identifies a similar word as akeyword if the similar word similar to the word is included in thekeyword dictionary. As a result, even when the word included in the newmessage D2 is not included in the keyword dictionary, the informationprocessing device 1 is capable of identifying a similar word as akeyword to be used in the model of estimating a forwarding destinationif the similar word similar to the word is included in the keyworddictionary. Therefore, according to the information processing device 1,even in the case where the word included in the unknown message withouta similar message in the past message storage unit 121 is an unknownword, it is possible to estimate a forwarding destination using theknown keyword similar to the word.

Furthermore, a plurality of keywords included in the keyword dictionaryof the keyword dictionary storage unit 122 is keywords used to createthe model stored in the forwarding destination estimation model storageunit 125. As a result, according to the information processing device 1,it is possible to identify a keyword based on the words included in thenew message D2 from among the keywords used to create the model forestimating a forwarding destination.

Furthermore, in the case of outputting the estimated forwardingdestination, the information processing device 1 outputs informationassociated with the identified keyword. As a result, the user is enabledto check the information associated with the identified keyword for usein the model for estimating a forwarding destination. For example, theuser is enabled to verify whether or not the forwarding destination isappropriately estimated on the basis of the information associated withthe identified keyword.

Note that each of the illustrated components in each of the devices isnot necessarily physically configured as illustrated in the drawings.For example, the specific aspects of distribution and integration of therespective devices are not limited to the illustrated aspects, and allor some of the devices may be functionally or physically distributed andintegrated in any unit in accordance with various loads, use status, andthe like.

For example, although the information processing device 1 according tothe present embodiment includes the learning unit 14, the learning unit14 may be implemented by another information processing device. In thiscase, the storage unit 12 stores data obtained during learning by theanother information processing device.

Furthermore, various processing functions of the message reception unit10, the estimation unit 11, the output unit 13, and the learning unit 14executed in the information processing device 1 may be entirely oroptionally partially executed in a central processing unit (CPU) (or amicrocomputer such as a microprocessor unit (MPU) or a micro controllerunit (MCU)) as an exemplary control unit. Furthermore, it is needless tosay that whole or any part of the various processing functions may beexecuted by a program to be analyzed and executed in a CPU (ormicrocomputer such as MPU or MCU) or in hardware by wired logic.Furthermore, various processing functions executed with the informationprocessing device 1 may be executed by a plurality of computers incooperation through cloud computing.

Meanwhile, the various types of processing described in the aboveembodiment may be implemented by execution of a prepared program in acomputer. Thus, hereinafter, an exemplary computer configuration(hardware) that executes a program having functions similar to the aboveembodiment will be described. FIG. 15 is an explanatory diagramillustrating an exemplary computer configuration.

As illustrated in FIG. 15, a computer 200 includes a CPU 201 thatexecutes various types of arithmetic processing, an input device 202that receives data input, a monitor 203, and a speaker 204. Furthermore,the computer 200 includes a medium reading device 205 that reads aprogram and the like from a storage medium, an interface device 206 tobe connected to various devices, and a communication device 207 to beconnected to and communicate with an external device in a wired orwireless manner. Furthermore, the information processing device 1further includes a random access memory (RAM) 208 that temporarilystores various types of information, and a hard disk device 209.Furthermore, each of the units (201 to 209) in the computer 200 isconnected to a bus 210.

The hard disk device 209 stores a program 211 for executing variouskinds of processing in the functional configuration (e.g., messagereception unit 10, estimation unit 11, output unit 13, and learning unit14) described in the above embodiment. Furthermore, the hard disk device209 stores various data 212 that the program 211 refers to. The inputdevice 202 receives, for example, an input of operation information froman operator. The monitor 203 displays, for example, various screensoperated by the operator. The interface device 206 is connected to, forexample, a printing device or the like. The communication device 207 isconnected to a communication network such as a local area network (LAN),and exchanges various types of information with an external device viathe communication network.

The CPU 201 reads the program 211 stored in the hard disk device 209,loads it in the RAM 208 and execute it, thereby performing various kindsof processing related to the functional configuration described above(e.g., message reception unit 10, estimation unit 11, output unit 13,and learning unit 14). Note that the program 211 may not be prestored inthe hard disk device 209. For example, the computer 200 may read theprogram 211 stored in a readable storage medium to execute it. Thestorage medium that is readable by the computer 200 corresponds to, forexample, a portable recording medium such as a compact disk read onlymemory (CD-ROM), a digital versatile disk (DVD), or a universal serialbus (USB) memory, a semiconductor memory such as a flash memory, a harddisk drive, or the like. Furthermore, the program 211 may be prestoredin a device connected to a public line, the Internet, a LAN, or thelike, and the computer 200 may read out the program 211 from the deviceto execute it.

All examples and conditional language provided herein are intended forthe pedagogical purposes of aiding the reader in understanding theinvention and the concepts contributed by the inventor to further theart, and are not to be construed as limitations to such specificallyrecited examples and conditions, nor does the organization of suchexamples in the specification relate to a showing of the superiority andinferiority of the invention. Although one or more embodiments of thepresent invention have been described in detail, it should be understoodthat the various changes, substitutions, and alterations could be madehereto without departing from the spirit and scope of the invention.

What is claimed is:
 1. A non-transitory computer-readable recordingmedium storing an estimation program that causes a processor included ina computer to execute a process, the process comprising: receiving amessage of an estimation target of a forwarding destination; searchingpast messages for a message similar to the received message; estimating,when there is no similar message among the past massage, the forwardingdestination of the message using a model of estimating the forwardingdestination based on a keyword that is identified based on a wordincluded in the received message; and outputting a forwardingdestination of the searched similar message or the estimated forwardingdestination.
 2. The non-transitory computer-readable recording mediumaccording to claim 1, wherein the estimating includes identifying theword as a keyword when the word is included in a keyword dictionary inwhich a plurality of keywords is set.
 3. The non-transitorycomputer-readable recording medium according to claim 2, wherein theestimating includes identifying the similar word as a keyword when theword is not included in the keyword dictionary and a similar wordsimilar to the word is included in the keyword dictionary.
 4. Thenon-transitory computer-readable recording medium according to claim 2,wherein the plurality of keywords included in the keyword dictionary isa keyword used to create the model.
 5. The non-transitorycomputer-readable recording medium according to claim 1, wherein theoutputting includes outputting information associated with theidentified keyword in a case of outputting the estimated forwardingdestination.
 6. An estimation method comprising: receiving a message ofan estimation target of a forwarding destination; searching pastmessages for a message similar to the received message; estimating, whenthere is no similar message among the past massage, the forwardingdestination of the message using a model of estimating the forwardingdestination based on a keyword that is identified based on a wordincluded in the received message; and outputting a forwardingdestination of the searched similar message or the estimated forwardingdestination.
 7. The estimation method according to claim 6, wherein theestimating includes identifying the word as a keyword when the word isincluded in a keyword dictionary in which a plurality of keywords isset.
 8. The estimation method according to claim 7, wherein theestimating includes identifying the similar word as a keyword when theword is not included in the keyword dictionary and a similar wordsimilar to the word is included in the keyword dictionary.
 9. Theestimation method according to claim 7, wherein the plurality ofkeywords included in the keyword dictionary is a keyword used to createthe model.
 10. The estimation method according to claim 6, wherein theoutputting includes outputting information associated with theidentified keyword in a case of outputting the estimated forwardingdestination.
 11. An information processing device comprising: a receiverconfigured to receive a message of an estimation target of a forwardingdestination; and a processor configured to: search past messages for amessage similar to the received message, estimate, when there is nosimilar message among the past massage, the forwarding destination ofthe message using a model of estimating the forwarding destination basedon a keyword that is identified based on a word included in the receivedmessage, and output a forwarding destination of the searched similarmessage or the estimated forwarding destination.
 12. The informationprocessing device according to claim 11, wherein the processoridentifies the word as a keyword when the word is included in a keyworddictionary in which a plurality of keywords is set.
 13. The informationprocessing device according to claim 12, wherein the processoridentifies the similar word as a keyword when the word is not includedin the keyword dictionary and a similar word similar to the word isincluded in the keyword dictionary.
 14. The information processingdevice according to claim 12, wherein the plurality of keywords includedin the keyword dictionary is a keyword used to create the model.
 15. Theinformation processing device according to claim 11, wherein theprocessor outputs information associated with the identified keyword ina case of outputting the estimated forwarding destination.